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ABSTRACT OF THE DISCLOSURE 

An automated multipurpose, real -time apparatus and procedure 
is provided herein for administration of at least one of visual 
and auditory stimuli to a patient for the purpose of testing at 
least one of visual, auditory, attentional and cognitive per- 
formance. The apparatus includes at least one of visual and 
auditory stimulus generation means, response acquisition means, 
data analysis, archiving and display means and host computer 
means with memory and mass storage capability. Stimulus 
generation means, including at least one of auditory and visual 
stimulus generation means is used to present calibrated, prede- 
termined sequences of test stimuli to the subject under control 
of the host computer. Both test stimuli and test instructions 
may be presented using the stimulus generation means. Subject 
response means are provided for registering behavioral responses 
from the subject and for relaying these responses to the host 
computer. 
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THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLOSIVE 
PROPEERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS: 



1. An interactive diagnostic system of the stimulus- 
responsive type for evaluating performance levels of a selected 
at least one of visual, auditory, cognitive, and attentional 
capacities of a subject, comprising: 

<a) a host computer; 

(b) generator and displaying means for presenting 

predetermined sequential stimuli including at least one 
of visual stimuli and auditory stimuli to said subject 
in response to selected stimuli modules in said host 
computer; 

<c) subject response means for registering responses of 
said subject to said sequential stimuli and for 
relaying said responses to said host computer; 

(d) data entry means for instructing said host computer to 
provide said selected test modules in a predetermined 
sequences ; and 

(e) and an analysis, storage and communication means to 
provide for the analysis of test data, archival storage 
of recently acquired test data, comparison with 
previously stored data and the reporting of test 
results. 



2. An automated, multi-purpose, real-time apparatus for 
administration of stimuli to a patient, for evaluating 
performance levels of a selected at least one of visual, 
auditory, cognitive and attentional capacities of a patient, 
using a host computer including memory and mass storage 
capabilities for storing data and instructions, said apparatus 
comprising: 

(a) an operator station means having an input display means 
permitting the system operator to select individual 
test parameters, individual tests or sequences of 
tests; 
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<b) a patient station means having a display means for 

stimulation o£ said patient, and a response means £or 
collection of patient responses to said host computer 
corresponding to stimuli used in a selected test; 

(c) stimulus generator module for providing visual auditory 
stimulation means from both real-time and mass storage; 

(d) a response data acquisition peripheral means for 
capturing patient responses; and 

(e) means for recording, archiving, analyzing and reporting 
patient responses. 

3. The apparatus of claim 2, wherein said response data 
acquisition peripheral means includes an eye position monitor for 
sensing eye position and movements. 

4. An apparatus according to claim 3, wherein said eye 
position monitor comprises an infra-red light source and an 
infra-red sensitive closed circuit television camera. 

5. An apparatus according to claim 5, wherein the image of 
said eye generated by said camera yields a position map of said 
eye. 

6. An apparatus according to claim 6, wherein said position 
map indicates current eye position within a forty-degree frontal 
field of vision by means of a bit matrix updated by the host 
computer. 
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This invention relates to a system for providing and 
administering visual and auditory tests to observers to assess at 
5 least one of their visual, auditory, cognitive and attentional 
performance capability, through measurement of behavioural and 
electrophysiological responses. 

People process information about their environment through 
the senses of vision and hearing. Vision and hearing senses 

10 involve the transduction of variations in photic and acoustic 
energy radiated or reflected from surrounding objects into 
nervous system signals. As such, these sensory systems also 
include the neural mechanisms used in the coding and transmission 
of sensory signals within the nervous system. Because of their 

15 importance in every aspect of our life, vision and auditory 
functions receive much attention in health care services. 

The neural substrate that mediates vision and hearing is 
widely distributed throughout the brain. Consequently, diseases 
and disorders that effect the brain will often result in visul 

20 and auditory abnormalities. Thus, information regarding vision 
and hearing is of interest and use to the medical diagnostician. 

The visual and auditory systems are important channels for 
providing input for the evaluation of certain aspects of 
cognition and attentional capabilities in patients. In most 

25 neurological and neuropsychological evaluation procedures the 
patient under examination is given a variety of stimulus- 
response-type tasks to perform and the nature of the responses 
obtained in these tasks provides diagnostically useful infor- 
mation. 

30 Rapid and noninvasive evaluation of visual, auditory, 

cognitive and attentional performance is also desirable in many 
non-medical applications. For example, there are many appli- 
cations where it is necessary to evaluate large numbers of 
individuals in order to detect those among them who may be 

35 especially well suited or especially poorly suited for performing 
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particular tasks requiring a given degree of visual, auditory, 
cognitive or attentionaJ capacity. Examples of these include 
screening for air traffic controllers or for operators of 
transportation equipment. 

Automation of such tasks makes possible significant 
increases in the quantity and the quality of the evaluation data 
that can be collected in a given period of time; while at the 
same time increasing test speed, patient comfort and decreasing 
costs. These are important advantages when considering, for 
example, the cost of testing of large numbers of individuals or 
when attempting to. monitor individuals at risk in industrial 
occupations where relatively subtle changes must be detected. As 
a person's cognitive and attentional capacities are known to vary 
with changes in the level of fatigue, arousal and the number of 
other tasks required of the subject at the same time, it is 
important to be able to evaluate these capabilities under con- 
ditions where all of the variables that affect the subjects 
perormance can. be monitored and, where possible, controlled in 
real time. 

Automation also makes possible the use of dynamic logic to 
branch within a single test and among various testa administered 
to an individual at speeds not possible using manual test 
administration and evaluation methods. Further, it makes pos- 
sible rapid comparison of the subject's response data to those 
obtained from one or more normative data sets in an on-line 
database. 

There are two main test types bsed upon the kind of 
responses obtained from the subject. In the first type 
behavioral responses are obtained from subjects by detecting 
specific voluntary muscle movements (of the hands, eyes and vocal 
apparatus) during the execution of certain tasks. In the second 
type, electrophysiological responses (heart rate, electrically 
evoked and endogenous responses from the brain, etc.) are 
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obtained from subjects during the execution of certain tasks as 
well; however, voluntary muscle movements may or may not 
5 accompany these responses. Thus the latter may provide useful 

information from patients where voluntary muscle movements may be 
difficult or impossible or where voluntary responses are suspect. 

Behavioural tests have the advantages of speed, accuracy, 
relative technical simplicity and non-invasiveness . Some of the 

10 most venerable and diagnostically useful tests of visual and 
auditory function, for exmple, are behavioural tests. These 
include the measurement of visual acuity, color vision, visual 
fields, dark adaptation, pure tone and speech audiometry. Many 
behavioral tests have been shown to be of use in following the 

15 progression and remission of diseases as well as in estimating 
the amoun-t of residual function after cerebrovascular accidents 
and trauma. They can also be used to provide objective evalua- 
tion of therapeutic interventions (for example, drug treatments) 
for a variety of disorders that affect vision, hearing, cognition 

20 and attention. 

Electrophysiological tests offer a somewhat more objective 
estimate of a limited set of sensory, cognitive and attentional 
functions. There are three areas of application where visual and 
auditory evoked electrophysiological responses complement the 

25 behavioral tests: (1) as an index of neural mechanisms not 
amenable to analysis through behavioral means; (2) for more 
objective confirmation of psychophysical findings in patients who 
are, for whatever reason, difficult to test; and (3) intraopera- 
tive monitoring of sensory status of patients under general 

30 anaesthetic. 

Although the past decade has shown the emergence of many 
devices for the analysis of electrical responses from the scalp 
(and which concentrate mainly, upon the signal processing aspects 
of the recording technique), few automatic methods have been 

35 developed that generate stimuli, control stimulus presentation, 

collect subject responses and store and manage the data obtained. 
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U.S. Patent No. 4,474,186 to Ledley et al , issued October 2, 
1984, describes a computerized electro-oculographic system with 
5 feedback control of stimuli for administration of electro- 
oculographic tests and visual evoked response tests to a patient, 
and for automated processing of results derived from such tests. 
Automated administration of the tests is conducted either under 
the direct control of the test administrator via an operator 

10 control section or under the automated control of a programmed 
computer with indirect control by the test administrator. It 
should be noted that such apparatus, although providing automated 
administration of certain test stimuli to a patient via employ- 
ment of an operator control console, does not have multi-purpose 

15 capability, nor does it provide comparison capability of test 
results with normative data from a database. 

Therefore, it is an object of a principal aspect of the 
present invention to provide a flexible, automated and multi- 
purpose, real-time based system for test administration and for 

20 recording acquired behavioral and electropsyciological responses 
in real-time. 

Accordingly, by one broad aspect of the present invention, 
an interactive, automated patient diagnostic system of the 
stimulus-response type is provided herein for evaluating aspects 

25 of at least one of visual, auditory, cognitive, and attentional 
performance and capacity of a human subject, comprising: (a) a 
host computer; <b) a stimulus generation and display means for 
presenting predetermined sequential stimuli including at least 
one of visual and auditory stimulus to the subject in response to 

30 selected stimuli modules in the host computer; (c> subject 

response means for registering responses of the human subject to 
the sequential stimuli and for transmitting the responses to the 
host computer; (d) data entry means for instructing the host 
computer to provide the selected test modules in a predetermined 

35 sequence; and (e) and analysis, storage communication and display 
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means to provide for the analysis of test data, comparison with 
previously stored data and the reporting of test results and 
5 analysis. 

In a specific embodiment of the apparatus of this aspect of 
the present invention, the apparatus comprises: (a) an operator 
station means having an input terminal and output display means 
permitting the system operator to select individual test 

10 parameters, individual tests or sequences of tests; (b) a patient 
station rneans having a display means for stimulating the patient 
with visual and auditory stimuli, and a response means for col- 
lection of patient responses to a host computer corresponding to 
the stimuli used in a selected test; (c) a stimulus generator 

15 module for providing a stimulation test means through both mass 
storage and real-time means; (d) a response data acquisition 
peripheral means for capturing patient responses; <e) means for 
recording patient responses; (f ) a data analysis means for 
analyzing the data obtained; and (g) database means for compari- 

20 son of an individual's test results with normative data stored in 
the database. 

The response data acquisition peripheral means includes a 
hand response monitor, a voice response monitor and an eye 
movement monitor for sensing and recording eye position and 

25 movements. The eye movement monitor preferably comprises an 

infra-red light source and an infra-red sensitive closed circuit 
television camera. 

The means for administering at least one of visual and 
auditory stimuli preferably includes both two visual stimulation 

30 means, and an auditory stimulation means. Preferably, the visual 
stimulation means includes a video display screen subtending a 
visual angle of about 40 degrees at a distance of about 2 metres 
from the patient and a graphics display terminal, placed within 
reaching distance of the patient. The auditory stimulation means 

35 includes a pair of ear speakers or headphones. 
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In general terms, the apparatus includes at least one of 
visual and auditory stimulus generation means, a response 
5 acquisition means for hand eye and voice, a data display means 
and a host computer with memory, mass storage capability and 
hardcopy capability. The stimulus generator means including at 
least one of auditory and visual stimulus generation means is 
used to present calibrated, predetermined sequences of test 

10 stimuli to the subject/patient under control of the host com- 
puter. Both test stimuli and test instructions may be presented 
using the stimulus generation means. Subject response means are 
provided for registering various behavioral responses from the 
subject and for relaying these responses to the host computer. 

15 The apparatus also includes an operator control means by which 
command and data entry are provided to the host computer. This 
provides the capability for modification of the specifics of 
individual test modules and for the assemblage of various 
selected test modules in a variety of predetermined sequences. 

20 The host computer provides data analysis and storage of the 
results obtained from the administration of tests including 
comparison means of individual test data to that of group 
normative data obtained in the same tests. 
In the accompanying drawings, 

25 FIGURE 1 is a block diagram of a computerized, real-time 

visual and auditory stimulator and test system according to one 
embodiment of the present invention showing its main components; 
and 

FIGURE 2 is a flow-chart explaining the operation of the 
30 system of FIGURE 1. 

Figure 1 of the drawings shows a block diagram of the 
computerized test system. It consists of a host computer 10, 
database 24 and hardcopy unit 17, and an operator console 11 
which includes a video display 12 and a keyboard 13. In 
35 addition, it consists of a patient station 14 which includes a 
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visual display 15 within reach of the patient for presenting 
stimuli and instructions, and a manual response entry terminal 16 
5 (15 and 16 are, preferably, embodied in a single touch screen 
display). The patient station also has a large format visual 
display 27 for presenting stimuli and instructions, (subtending a 
visual angle of about 40 degrees and placed at a distance of 2 
metres front the subject's eyes) and a pair of ear speakers or 

10 headphones 60 worn by the patient for presenting stimuli and 

instructions to the patient. Attached to the host computer is a 
stimulation subcomponent 18, which provides an interface between — 
the visual and auditory stimuli! devices of the patient station 
14 and the host computer 10. The real-time image generation 

15 component 19 provides images for presentation on the visual 

display 15 while the image mass storage subcomponent 20 provides 
visual stimuli to the visual display 27. The real-time sound 
generation subcomponent 25 and the sound mass storage subcom- 
ponent 26 provide auditory stimuli to the patient's headphones 

20 60. The behavioral response monitoring subcomponent 28 provides 
an interface for the acquisition of eye 23, hand 21 and voice 22 
responses by the host computer 10. 

Should it be also desirable to include, in the system, 
electrophysiology response monitoring, for example as taught by 

25 Ledley et al in United States Patent 4,474,186, electrical 
response monitoring means 29 would be necessary. 

With reference to Figure 2 of the drawings, the flow-chart 
explaining the overall system operation will now be described. 
After log-on 30 the operator may choose one of three modes via 

30 mode select 31. The three modes of operation are system 

development 32, operator training 33, and patient testing 34. 
The first two modes are self-explanatory. For example, the 
system development mode 32 is used to enter commands and data for 
new tests, and to establish new test batteries, and the like. It 

35 is also used to create specific databases to which incoming test 
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data will be compared. The operational mode proper is called the 
patient testing mode 34, which, when selected, displays a master 
5 menu 35, with at least three choices: either a menu 36 of single 
tests, a menu 37 of test batteries, or establishment of a new 
patient file 38. The latter, when selected, permits the operator 
to enter new patient data which may result in the updating of one 
or more of the files in the patient data base 39. Once the test 

10 batteries menu 37 has been displayed and a particular test 

battery "J" has been selected, the system retrieves the test 
files 40 of that battery and begins with the test at the top of 
the battery "J" file at next test 41 by executing that test's 
generation file 42, which commences a sequence of timed commands 

15 and signal timing triggers 43 to activate stimulus generation 
peripherals 44 (corresponding to 18 in Figure 1), and response 
data acquisition peripherals 45 (corresponding to 29 in Figure 
1). The collected data are analyzed at 46 in light of the 
stimulus generated, the patient data, and the test database (s) 

20 (that could include normative data), and a decision rule is used, 
to determine the validity of the test and data at decision block 
47. For example, if a large number of stimulus trials are not 
responded to within preestabl ished critical limits controlled by 
timing triggers 43, the test data are not considered as a valid 

25 measure of the function of under test and the test is then 

repeated 48. If the data and test administration are found to be 
valid, then the system invokes the next test generation file, and 
so on until the final test has been completed. The system nay 
also invoke an update of the test data base 49. The results are 

30 then archived, analyzed and displayed. After the last test is 
completed the results may be printed 51. The single test menu 
36, had it been selected, would simply enable the operator to 
retrieve a single test 50 and administer it in the same manner 
via the same routines. However, no linkage would be made to any 

35 other test upon completion. 
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The determination of the validity of individual responses in 
nay behavioral tests requires the evaluation of eye position 
5 and/or eye movement monitoring. In Figure 1, a preferred sub- 
system for the acquisition of eye position is shown. The host 
computer 10, upon executing a given test generation file, con- 
trols the generation of visual images to be presented to either 
to the touch screen display 15 or to the large format visual 

10 display 27. The host computer 10 then retrieves information 
about the position of the subject's eye from the eye position 
subcomponent 23. An infra-red sensitive camera with a telephoto 
lens 62 is focussed upon the pupil of the subject's right eye 
while the eye is illuminated with low-level infra-red radiation 

15 through the use of an infra-red illuminator 61. The output of 

the camera is the spatial characteristics of the infra-red image 
of the pupil and this information 64 is passed to the host 
computer 10. The current eye position 64 is resolved to within 
about 0.25 degrees about the 40 degree field of view, which is 

20 sufficient, for the purposes of this system, to monitor direction 
of gaze during testing. It is possible, using this system, to 
edit out of the test data trials in real-time those where the 
eyes were not directed in the appropriate position. 

An example showing interaction with the system for a single 

25 psychophysical test will now be described to illustrate the 
procedures generally common to many such tests, which differ 
mainly in the details of the stimuli presented and in the nature 
of the operator's and patient's instructions. 

30 (OPERATOR SCREEN! 



{Hospital Laboratory Name J Operator ID: [Name] 

35 Name of Patient Currently on System: [Namel 
Patient File ID [xxjcxi 

Time: xx xx xx Date: xx xx xx 

40 
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SINGLE TEST MENU 

5 

Psychophysical Tests: 
1 Variable-contrast Snellen acuity tests 

2 Spatiotemporal contrast sensitivity tests 

10 

3 Bye movement control system analysis tests 

4 Pure tone audiometry tests 

15 5 ..Dichotic listening tests 

6 Static and dynamic perimetry tests 

7 Complex visual pattern recognition tests 

20 

8 Driver vision test 

(Optional) Evoked Potential Tests 

25 9 Stimulus-Evoked Cortical Potential tests 

10 Stimulus-Evoked Blectroretinogram tests (ERG) 

30 



PLEASE ENTER CHOICE > 2 < RET > TO MASTER MEND 

35 , ] 

For this example, the operator has decided to administer one 
o£ the spatiotemporal contrast sensitivity tests <#2). Another 
menu comes onto the operator's screen offering the various 

40 versions of this test. From this menu onward throughout the 

testing session two additional control options become available 
to the operator. (1) < RETURN > to invoke default choices to 
individual questions and (2) < ESC > to invoke default options 
for entire sections of questions during the set up procedure . 

45 The setup file contains information regarding the names of 

the stimulus graphics files used in the particular test under 
consideration, control parameters used for modifying stimulus 
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characteristics depending upon the patient's responses. In 
addition, parameters for patient performance optimization can be 
5 provided in this f ile--including the use of eye position moni- 
toring, feedback to the patient regarding test performance, and 
arousal and attention regulation through the use of visual or 
auditory signals. It also contains the file names of two 
additional files — one for text for patient instructions and 
10 prompts and one for operator prompts. 

[OPERATOR SCREEN] 



Test #2 

15 Spatiotemporal Contrast Sensitivity Test 

Enter Test Initialization filename:. RETURN 

(Default) 

20 Enter Patient Instruction Set filename: RETURN 

(Default) 

Enter Operator Instruction Set filename: RETURN 

(Default) 

25 

Initial Operator Instructions: 

1. Position Patient in testing booth. Ensure that patient's 
30 glasses do not interfere with eye-novement monitor and are 

comfortable. Adjust seat so patient can reach the touch screen. 



35 10 MINUTES OF DARK ADAPTATION IS REQUIRED PRIOR 

TO THIS TEST 



Copied from l<)4822f)8 on 12/12/2()<)7 



12 - 



2. Press RETURN when ready to deliver patient instruction set: 

< RETURN > 

subject Instructions Page 1 now presented on patient monitor. 
Press RETURN when ready to begin adaptation. 

< RBTDRN > 

Pre-Adaptation in progress (X minutes remaining) 

^^-^ operator chose to use the default filenames 

but could have entered others, invoking "customieed" files. The 
instruction file is written in "pages" and contains special 
characters (transparent to the instruction reader) that delimit 
the amount of information presented on a single "page" and link 
these xnstructions to graphics examples. 

The patient display monitor always has some type of pattern 
present-except when darkness is required for a test. This is 
true even when a test is not being administered. Audio infor- 
mation xs presented during waiting periods to help maintain 
patient arousal levels. 

The subject's instructions appear on the patient display 
monxtor and on the operator's monitor. At this point feedback 
from the patient is useful regarding comprehension of the 
instructions. The patient instruction set is always linked to a 
short dry run" of the experiment. An example dry run (with 
patxent responses in brackets) follows: 
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[PATIENT SCREEN! 



Welcome to the 
[(Hospital Laboratory Name)] 
In this test there are two windows on the screen that you 
are to watch. One window has a pattern in it while the other is 
blank. Please press the left button on the touch screen if you 
see the pattern on the left side and the right button if you see 
the pattern on the right. if you don't see the pattern on either 
side, try to guess. 

Here is an example: 

[the test pattern appears in left window] 

"In this case, the pattern is seen in the left window so you 
press the left button. PRESS THE LEFT BUTTON NOW. 

(button is pressed by patient) 

"Correct. Get ready for the next one." 

(after a few seconds) 
"This ends the practice runs . " 

If there is d^rk adaptation before the test, something like the 
following will be presented; 

"Please wait for a few minutes to adapt your eyes" 
At this point an operator prompt will start the adaptation 
session and a timed signal will alert the patient near the end of 
the adaptation session. 
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After the adaptation period is completed, the following prompt 
will be presented through the headphones and on the video screi 

"Test begins in 10 seconds. Watch the screen and respond 
when requested to do so in the same way as in the example 
you just carried out." 



At this point the test begins. 



The actual testing task in this case, consists of a large 
number of trials of monocular viewing (i.e., each eye tested 
separately) of a sinusoidal contrast grating of fixed spatial 
frequency and orientation presented in only one of two "window 
areas" on the video graphics display. On each trial the patient 
must decide in which of the window areas (say, in this example, 
the left or the right) a pattern is present. The orientation, 
rate of temporal modulation, spatial frequency and luminance 
contrast of the pattern is varied randomly as is the location of 
presentation of the pattern on each trial . 

After every stimulus trial is presented, the patient is 
provided with feedback (typically, auditory feedback for visual 
tests and vice versa) on whether or not his response was correct. 
This aids in maintaining arousal level and interest by providing 
a running estimate to the patient of his performance somewhat in 
the manner of an arcade game. 

Threshold for detection of the "pattern" is estimated for 
each of the parameter ranges that are randomly sampled in the 
test: spatial frequency, orientation, temporal frequency; with 
spatial modulation (i.e., contrast) of the pattern being the 
dependent variable. 

During the test a number of measures are taken that are 
relevant to the determination of the patient's visual sensitivity 
and the evaluation of the quality of data obtained from the 
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patient. By monitoring eye position during stimulus presentation 
it is possible to know if the patient is looking at the pattern 
or not, and by monitoring patient response tiroe within and 
between test trials and comparing these data to previously 
obtained response times from the patient, an estimate of the 
level of fatigue at that point in the test can be obtained. 
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